Natural Heuristic Dynamic Programming for Dynamic Systems

Heuristic Dynamic Programming (HDP) is the simplest kind of Adaptive Critic 1]. It can be used to maximize or minimize any utility function, such as total energy or trajectory error, of a system over time in a noisy environment. In this article, we propose a new version of HDP, called NHDP (Natural Heuristic Dynamic Programming). This new version incorporates basic HDP algorithm with the following features: (i) use of Trajectory Pattern Method to guarantee smoothness of trajectory and control signals; (ii) use multiple critic networks to localize eeect of each parameter mimicing the natural biological model of human brain; and (iii) allow the controller to learn from slow to fast motion, analogous to the natural learning behavior of humans. A simple dynamic system is used to illustrate NHDP.